Non-same camera based image processing apparatus

ABSTRACT

The present invention provides an image processing apparatus comprising: a first camera obtaining a true-color image by capturing a subject; a second camera spaced apart from the first camera and obtaining an infrared image by capturing the subject; and a control unit connected to the first camera and the second camera, wherein the control unit matches the true-color image and the infrared image and obtains three-dimensional information of the subject by using the matched infrared image in a region corresponding to the matched true-color image and a valid pixel.

TECHNICAL FIELD

The present disclosure relates to an image processing apparatus based onnon-identical cameras. Particularly, the present disclosure isapplicable to a technical field for extracting either or bothtwo-dimensional information and three-dimensional information based on ared, green, and blue (RGB) camera and an infrared camera.

BACKGROUND ART

Humans are capable of seeing things three-dimensionally based ondisparity between the two eyes. In recent years, a stereo vision camerathat calculates distance information with two cameras according to sucha principle has been widely used.

FIG. 1 shows an embodiment of a conventional stereo vision camera.

The stereo vision camera may include two cameras 101 and 102 spacedapart from each other by a predetermined distance d1. The stereo visioncamera may capture a subject 300 that exists in an area 500 captured bythe two cameras, and calculate disparity between images captured by thetwo cameras, and obtain a distance d2 from the cameras 101 and 102 tothe subject 300.

To measure the distance to the subject based on the disparity, epipolargeometry, which is a well-known technique, may be employed.

However, since the conventional stereo vision camera matches two imagesbased on the same camera, it is difficult to overcome limitations fromthe use of a single camera type (single band).

For example, stereo vision based on red, green, and blue (RGB) camerashas a problem in visibility at night, and stereo vision based oninfrared cameras has a problem in color recognition.

Therefore, a dual-band stereo vision system has been introduced. Thedual-band stereo vision system is a stereo vision system that includesboth the stereo vision based on RGB cameras and the stereo vision basedon infrared cameras.

However, the dual-band stereo vision system has disadvantages in that animage processing apparatus has a large volume and weight, high cost isrequired, and it is difficult to match images acquired in each band.

DISCLOSURE Technical Problem

The object of the present disclosure is to provide an image processingapparatus based on non-identical cameras. Specifically, the object ofthe present disclosure is to obtain complementary two-dimensionalinformation from each camera and obtain three-dimensional informationfrom matched image information.

Technical Solution

In an aspect of the present disclosure, an image processing apparatus isprovided. The image processing apparatus may include a first cameraconfigured to capture a subject and obtain a true color image, a secondcamera spaced apart from the first camera and configured to capture thesubject and obtain an infrared image, and a controller connected to thefirst camera and the second camera. The controller may be configured tomatch the true color image and the infrared image and obtainthree-dimensional information about the subject in an area correspondingto effective pixels of the matched true color image from the matchedinfrared image.

The effective pixels may be pixels that absorbs light more than or equalto a minimum amount of light for identifying an object and less than orequal to a saturation amount of light

The controller may be configured to coordinate a representative pixel ofthe subject among the effective pixels to the infrared image and obtainthe three-dimensional information based on stereo vision operation.

The representative pixel may include at least one of an outline pixel ofthe subject and a center pixel of the subject.

When a shape of a specific object is recognized in the areacorresponding to the effective pixels, the controller may be configuredto detect whether the shape of the specific object actually correspondsto the specific object based on information about a related area in thematched infrared image.

The controller may be configured to obtain two-dimensional informationabout an area corresponding to abnormal pixels of the matched true colorimage from the matched infrared image.

The two-dimensional information may include information about presenceof a specific object in the area corresponding to the abnormal pixels.

When the presence of the specific object is detected from thetwo-dimensional information, the controller may be configured to convertthe abnormal pixels into effective pixels by adjusting an aperture ofthe first camera.

After adjusting the aperture of the first camera, the controller may beconfigured to obtain three-dimensional information about the specificobject present in the area corresponding to the abnormal pixels.

When the presence of the specific object is detected from thetwo-dimensional information, the controller may be configured to convertthe abnormal pixels into effective pixels based on raw data forobtaining the true color image and obtain three-dimensional informationabout the specific object present in an area corresponding to theconverted effective pixels.

The image processing apparatus may include an output interface. When thepresence of the specific object is detected from the two-dimensionalinformation, the controller may be configured to output informationabout the specific object to a user through the output interface.

The output interface may include a display module configured to displaythe true color image, and the controller may be configured to overlapand display an infrared image of the specific object in the areacorresponding to the abnormal pixels.

The first camera may be a camera configured to detect visible light andnear-infrared (NIR) light.

The second camera may be a camera configured to detect at least one offar-infrared (FIR) light, mid-wave infrared (MWIR) light, orshort-wavelength infrared (SWIR) light.

Advantageous Effects

The present disclosure provides a solution for obtaining complementarytwo-dimensional information based on non-identical cameras.

The present disclosure provides a solution for obtainingthree-dimensional information based on non-identical cameras.

The additional applicability of the present disclosure will becomeapparent from the detailed description given hereinafter. However, itshould be understood that the detailed description and specific examplessuch as preferred embodiments of the present disclosure are given by wayof illustration only, since various changes and modifications within thespirit and scope of the present disclosure will become apparent to thoseskilled in the art from this detailed description.

DESCRIPTION OF DRAWINGS

FIG. 1 shows an embodiment of a conventional stereo vision camera.

FIG. 2 shows an embodiment of a non-identical stereo vision cameraaccording to the present disclosure.

FIG. 3 is a flowchart illustrating a method of acquiring two-dimensional(2D) information and three-dimensional (3D) information through thenon-identical stereo vision camera according to the present disclosure.

FIG. 4 shows matching of true color and infrared images obtained by thenon-identical stereo vision camera according to the present disclosure.

FIGS. 5 and 6 are views for explaining effective and abnormal pixels ofa true color image obtained by the non-identical stereo vision cameraaccording to the present disclosure.

FIG. 7 is a view for explaining a method of obtaining 3D informationfrom a true color image and an infrared image acquired by thenon-identical stereo vision camera according to the present disclosure.

FIG. 8 is a view for explaining a method of identifying a specificobject when the specific shape exists in a true color image obtained bythe non-identical stereo vision camera according to the presentdisclosure.

FIG. 9 is a view showing an embodiment in which a specific object isemphasized and output based on an infrared image when the specificobject is detected from a true color image obtained by the non-identicalstereo vision camera according to the present disclosure.

BEST MODE

Hereinafter, embodiments of the present disclosure will be described indetail with reference to the attached drawings. In this specification,the same or equivalent components will be provided with the samereference numbers, and description thereof will not be repeated. Thesuffixes “module” and “unit” of elements herein are used for convenienceof description and thus may be used interchangeably and do not have anydistinguishable meanings or functions. If it is determined that detaileddescriptions of the related art obscure the gist of the presentdisclosure, the detailed descriptions will be omitted. It should also beunderstood that the attached drawings are merely to provide betterunderstanding of the embodiments of the present disclosure and thespirit of the present disclosure is not limited to the attacheddrawings. Thus, the present disclosure should be construed to extend toany alterations, equivalents and substitutes in addition to those whichare particularly set out in the accompanying drawings.

The following embodiments of the present disclosure are intended toembody the present disclosure, not limiting the scope of the presentdisclosure. What could easily be derived from the detailed descriptionof the present disclosure and the embodiments by those skilled in theart is interpreted as falling within the scope of the presentdisclosure.

The above embodiments are therefore to be construed in all aspects asillustrative and not restrictive. The scope of the disclosure should bedetermined by the appended claims and their legal equivalents, not bythe above description, and all changes coming within the meaning andequivalency range of the appended claims are intended to be embracedtherein.

FIG. 2 shows an embodiment of a non-identical stereo vision cameraaccording to the present disclosure.

The non-identical stereo vision camera according to the presentdisclosure may include two non-identical cameras configured to detectdifferent wavelength bands.

One of the two non-identical cameras may be a first camera 201configured to obtain a true color image by capturing a subject 300. Theother may be a second camera 202 spaced apart from the first camera 201by a predetermined distance d1 and configured to obtain an infraredimage by capturing the subject 300.

Specifically, the first camera 201 may be a camera that detects visiblelight or a camera that detects visible light and near-infrared (NIR)light. That is, the true color image may be an image obtained bydetecting visible light, or an image obtained by detecting visible lightand NIR light. Here, the NIR light refers to light with a wavelengthbetween about 0.7 and 1.0 um.

The first camera 201 may include a complementary metal-oxidesemiconductor (CMOS) image sensor (CIS) that detects visible light.Alternatively, the first camera 201 may be a camera with high visibilityby extending a detection wavelength from visible light to NIR light.

The second camera 202 may be a camera that detects far-infrared (FIR)light, or a camera that detects mid-wave infrared (MWIR) light orshort-wavelength infrared (SWIR) light. Here, the FIR light refers tolight with a wavelength between about 8 and 14um, the MWIR light refersto light with a wavelength between about 3 and 5 um, and the SWIR lightrefers to light with a wavelength between about 1.0 and 2.5 um.

A camera that detects FIR light and SWIR light shows excellentperformance in identifying day/night pedestrians, and thus it may beused as night vision equipment.

The field of view (FOV1) of the first camera 201 may not need to be thesame as the field of view (FOV2) of the second camera 202. However, whenthe two cameras have different FOVs, image information needs to becompared and matched based on singularities of the acquired images.Details will be described later with reference to FIG. 4.

When the subject 300 exists in a first area 501 captured by both thefirst camera 201 and the second camera 202, a distance d2 from thenon-identical stereo camera to the subject 300 may be calculated basedon disparity between the images obtained by the first camera 201 and thesecond camera 202.

The distance d2 to the subject 300 may be obtained based on stereocalculation as follows. Pixels of the subject 300 are extracted fromboth the true color image obtained by the first camera 201 and theinfrared image obtained by the second camera 202, and then informationabout the distance d2 may be obtained based on disparity of thecorresponding pixels.

Here, the pixel may be a center pixel or at least one of pixelsconstituting the outline of the subject 300. Details will be describedlater with reference to FIG. 7.

In a second area 502 captured by only one of the first camera 201 andthe second camera 202, two-dimensional (2D) information may be obtainedfrom the photographing camera. FIG. 2 shows an embodiment in which thesecond area 502 is photographed only by the second camera 202, and inthis case, only 2D information included in the infrared image may beacquired from the second area 502. In contrast, when the second area 502is captured only by the first camera 201, only 2D information includedin the true color image may be acquired from the second area 502.

For the first area 501, the second camera 202 may obtain complementary2D information when no 2D information is obtained by the first camera101. Alternatively, the second camera 202 may be used to check 2Dinformation acquired by the first camera 101. Details will be describedlater with reference to FIG. 3.

FIG. 3 is a flowchart illustrating a method of acquiring 2D informationand three-dimensional (3D) information through the non-identical stereovision camera according to the present disclosure.

Specifically, FIG. 3 is a flowchart illustrating a method of acquiring2D information and 3D information through the non-identical stereovision camera according to the present disclosure shown in FIG. 2.

The non-identical stereo vision camera according to the presentdisclosure may acquire image data from the first camera 201 and thesecond camera 202 (S301). The first camera 201 may be a red, green, andblue (RGB) camera or a camera with an extended wavelength which iscapable of detecting NIR light. The second camera 202 is an infraredcamera that detects infrared wavelengths or a camera that detects FIRlight or SWIR light.

Since the first camera 201 and the second camera 202 are differentcameras, the image data obtained thereby may need to be matched (S302).This is a process of matching corresponding pixels of the image data.The matching process may be based on the singularity of each image data.

According to the present disclosure, 3D information or 2D informationmay be obtained depending on pixel values of the image data obtained bythe first camera 201. Here, the 2D information may includeidentification information about moving objects such as pedestrians andvehicles and identification information about fixed background objectssuch as buildings and traffic lights. Additionally, the 2D informationmay include text information for identifying signals and signs.

Specifically, when the pixel values of the image data obtained by thefirst camera 201 are valid (YES in S303), 3D information may be obtainedfrom the image data acquired by the second camera 202.

On the other hand, when the pixel values of the image data obtained bythe first camera 201 are not valid (NO in S303), 2D information may beobtained from the image data acquired by the second camera 202.

In this case, whether the pixel values of the image data acquired by thefirst camera 201 are valid may be checked based on the amount of lightabsorbed by pixels. That is, when the amount of light absorbed by apixel is more than or equal to the minimum amount of light foridentifying objects and less than or equal to the saturation amount oflight, the pixel may be considered to be effective.

Since the first camera 201 acquires true color images, the amount oflight becomes an important factor in identifying objects. For a truecolor image, it is difficult to identify objects in an extremely brightarea (i.e., an area corresponding to light saturation pixels) or in anextremely dark area (i.e., area corresponding to pixels less than orequal to the minimum amount of light). Here, the amount of light meansthe amount of light in visible light bands NIR light bands.

On the other hand, since the second camera 202 acquires infrared images,the second camera 202 has lower dependence on visible light illuminancein identifying objects than the first camera 201. In particular, a FIRcamera may use a passive device that requires no external light sources,and an MWIR or SWIR camera may obtain a clear image even when thevisible light illuminance is low or use an extra transmitter to obtain aclear image. Therefore, such cameras may be used to identify objectsinstead of the first camera 201 in the light saturation area or an areawhere the amount of light is less than or equal to the minimum amount oflight.

Thus, when the subject 300 exists in an area corresponding to effectivepixels of the image data acquired by the first camera 201, the subject300 may be identified by the first camera 201 and the second camera 202,and information about the distance to the subject 300, i.e., 3Dinformation may be obtained based on binocular disparity between the twocameras.

However, if the subject 300 exists in an area corresponding to abnormalpixels of the image data acquired by the first camera 201, the subject300 may not be identified by the first camera 201. Thus, the secondcamera 202 is used to identify the subject 300. In this case, it isdifficult to obtain 3D information because the subject 300 is identifiedby one camera, but it may be useful in that complementary 2D informationis obtained.

Similarly, when the subject 300 is not identified based on the imagedata acquired by the second camera 202, the first camera 101 may be usedto identify the subject 300. That is, complementary 2D information aboutthe image data acquired by the second camera 202 may be obtained by thefirst camera 101.

FIG. 4 shows matching of true color and infrared images obtained by thenon-identical stereo vision camera according to the present disclosure.

FIG. 4(a) shows an embodiment in which the non-identical stereo visioncamera according to the present disclosure is provided in a vehicle 400,and FIG. 4(b) shows matching of image data obtained by the stereo visioncamera according to the present disclosure while the vehicle 400 isdriving.

Specifically, FIG. 4(b) shows superimposition of a true color image 2011acquired by the first camera 201 and an infrared image 2021 acquired bythe second camera 202. In FIG. 4(b), the true color image 2011 and theinfrared image 2021 are matched to visually represent image datacorresponding to the first area 501 and image data corresponding to thesecond area 502, but the image shown in FIG. 4 is not provided to auser.

The true color image 2011 and the infrared image 2021 may be matchedbased on singularities of subjects 301 and 302. However, since the firstcamera 201 and the second camera 202 are spaced apart by a predetermineddistance and capture the subjects 301 and 302, disparity may be presentbetween the matching singularities.

FIG. 5 shows an embodiment in which the FOV of the first camera 201 islarger than the FOV of the second camera 202. For the second area 502captured only by the first camera 201, only 2D information may beobtained from the true color image 2011 acquired by the first camera201.

Specifically, a signboard 303 in the second area 502 may be identifiedbased on the true color image 2011 acquired by the first camera 201, andtext information included in the signboard 303 may be extracted.

3D information about the subjects 301 and 302 may be obtained from thefirst area 501 captured by both the first camera 201 and the secondcamera 202.

Specifically, a traffic light 301 and a vehicle 302 may be identified,and distance information may also be obtained. In addition,complementary 2D information may be obtained by each camera. Forexample, the color of the traffic light 301 may be identified from thetrue color image 2011 obtained by the first camera 201, and additionalinformation for identifying the vehicle 302 may be obtained from theinfrared image 2021 acquired by the second camera 202.

However, in some cases, it may be difficult to identify the subjects 301and 302 in the first area 501 through the first camera 501. That is, ifthe subjects 301 and 302 are in an area that is too dark or too bright,it may be difficult to identify the subjects 301 and 302 through thefirst camera 201.

Therefore, the present disclosure proposes the following technical idea:classifying effective pixels based on the amount of light in a truecolor image acquired from the first camera 201, obtaining 3D informationin an area corresponding to the effective pixels, and obtainingcomplementary 2D information in an area corresponding to abnormalpixels.

FIGS. 5 and 6 are views for explaining effective and abnormal pixels ofa true color image obtained by the non-identical stereo vision cameraaccording to the present disclosure.

When the vehicle 400 is driving at night, effective pixels of the truecolor image 2011 obtained by the first camera 201 may vary depending onnot only the reach of headlight beams 401, 402, and 403 but also ambientlighting as shown in FIG. 5(a).

FIG. 5(b) shows the true color image 2011 obtained by the vehicle 400 incorrespondence with FIG. 5(a). For convenience of description, the truecolor image 2011 shown in FIG. 5(b) is regarded as an imagecorresponding to the first area 501.

Referring to FIG. 5(b), the true color image 2011 includes an area 2013in which no objects are identified due to an insufficient amount oflight and an area 2012 in which objects are identified due to a properamount of light. The area 2013 in which no objects are identified andthe area 2012 in which objects are identified may not overlap with eachother and be included in the first area 501.

The area 2012 having the amount of light suitable for identifyingobjects may correspond to effective pixels, and 3D information about thesubject 300 in the area 2012 may be obtained from the second camera 202.

The area 2013 in which no objects are identified due to the insufficientamount of light may correspond to abnormal pixels, so that it isdifficult to obtain 2D information through the first camera 201. Thus,in the area 2013, complementary 2D information may be acquired by thesecond camera 202.

For example, when there is a pedestrian in the area 2013 in which noobjects are identified due to the insufficient amount of light, thepedestrian may be recognized by the second camera 202.

FIG. 6 shows an embodiment in which the area 2013 where no objects areidentified occurs in the true color image 2011 due to light saturation.

When a headlight beam 401 of a vehicle 400b driving in the oppositedirection is strong or when backlight or ambient lighting is strong,objects may not be identified in the true color image 2011 due to thelight saturation.

For the area 2013 in which no objects are identified due to the lightsaturation, 2D information may be obtained by the second camera 202.

In other words, the non-identical stereo vision camera according to thepresent disclosure may complementarily obtain 2D information, which maynot be obtained by the first camera 201 when the amount of light is lessthan or equal to the maximum level or more than or equal to thesaturation level.

It may also be considered that abnormal pixels are converted intoeffective pixels by adjusting the amount of light in an areacorresponding to the abnormal pixels.

To this end, when it is determined by the second camera 202 that aspecific object exists in the area 2013 corresponding to abnormalpixels, the area 2013 corresponding to abnormal pixels may be convertedinto the area 2012 corresponding to effective pixels by adjusting theaperture of the first camera 201.

When the area 2013 corresponding to abnormal pixels is converted to thearea 2012 corresponding to effective pixels by the adjustment of theaperture, 3D information about the specific object in the converted areamay be obtained from the infrared image 2012 acquired by the secondcamera 202.

When it is determined by the second camera 202 that a specific objectexists in the area 2013 corresponding to abnormal pixels, thecorresponding area 2013 may be converted into the area 2012corresponding to effective pixels based on raw data of the obtained truecolor image.

When the area 2013 corresponding to abnormal pixels is converted to thearea 2012 corresponding to effective pixels based on the raw data, 3Dinformation about the specific object in the converted area may beobtained from the infrared image 2012 acquired by the second camera 202.

Hereinafter, a method of acquiring 3D information (distance information)about the subject 300 from the true color image 2011 obtained by thefirst camera 201 and the infrared image 2012 obtained by the secondcamera 202 will be described.

FIG. 7 is a view for explaining a method of acquiring 3D informationfrom a true color image and an infrared image acquired by thenon-identical stereo vision camera according to the present disclosure.

The non-identical stereo vision camera according to the presentdisclosure may extract a representative pixel of the subject 300 from anarea corresponding to effective pixels of the true color image 2011,match the true color image 2011 and the infrared image 2012 based on therepresentative pixel, and obtain 3D information about the subject 300through stereo vision operation.

The representative pixel may be one of pixels constituting the outlineof the subject 300 or one of pixels constituting a frame 300a of thesame shape surrounding the subject 300. In some cases, therepresentative pixel may be a center pixel 300b having a minimumdistance difference to the pixels constituting the outline.Alternatively, the center pixel 300b may be selected by giving differentweights to distances to pixels representing the appearance of thesubject.

The distance to the subject 300 may be calculated based on the binoculardisparity of the representative pixel.

The non-identical stereo vision camera according to the presentdisclosure may identify the subject 300 based on data generated bymachine learning. According to the present disclosure, the distance to amain object may be discerned and calculated.

In the present disclosure, the infrared image 2012 acquired by thesecond camera 202 may be used to identify an object in an areacorresponding to effective pixels. Hereinafter, a method of additionallyidentifying an object from the infrared image 2012 acquired by thesecond camera 202 will be described.

FIG. 8 is a view for explaining a method of identifying a specificobject when the specific shape exists in a true color image obtained bythe non-identical stereo vision camera according to the presentdisclosure.

When the true color image 2011 includes only effective pixels, 2Dinformation may be easily obtained by the first camera 201.

In the true color image 2011, the shape of a subject may be defined bydistinguishing colors and singularities, and an object corresponding tothe shape may be identified.

However, as illustrated in FIG. 8, when the shape is a picture or asignboard, it may be difficult to distinguish whether an actual objectexists through the first camera 201.

That is, if it is determined by the first camera 201 that there are aperson 301 and an animal 302, the determination may provide incorrectinformation to the user. Therefore, whether an object recognized by thefirst camera corresponds to an actual object may be confirmed based on2D information acquired by the second camera 202.

For example, even if the shape of the person 301 is checked by the firstcamera 201, it may be determined based on heat distribution informationobtained by the second camera 202 that the shape does not correspond toan actual person.

FIG. 9 shows an embodiment in which a specific object is emphasized andoutput based on an infrared image when the specific object is detectedfrom a true color image obtained by the non-identical stereo visioncamera according to the present disclosure.

When the subject 300 exists in an area corresponding to effective pixelsof the true color image 2011, the non-identical stereo vision cameraaccording to the present disclosure may extract information about thedistance to the subject 300 from the infrared image 2012.

The extracted distance information may be provided to the user throughan output interface.

When the subject 300 exists in an area corresponding to abnormal pixelsof the true color image 2011, the non-identical stereo vision cameraaccording to the present disclosure may identify the shape of thesubject 300 from the infrared image 2012 and output information aboutthe specific object to the user through the output interface. To outputthe information about the specific object, a warning message may beprovided through at least one of a speaker and a display.

The non-identical stereo camera according to the present disclosure mayprovide at least one of the true color image 2011 and the infrared image2012 on a display. Alternatively, the non-identical stereo camera maycombine and provide the two images.

For example, when the subject 300 exists in an area corresponding toabnormal pixels of the true color image 2011, the non-identical stereovision camera may check the subject 300 in the infrared image 2012 andthen overlap and display a part of the infrared image 2012 by matchingit to the area.

Alternatively, even when the subject 300 exists in an area correspondingto effective pixels of the true color image 2011, if it is difficult forthe user to identify the subject 300 due to the amount of light, thenon-identical stereo vision camera may overlap and display a part of theinfrared image including the subject 300.

Specifically, FIG. 9(a) shows the true color image 2011 acquired by thefirst camera 201, FIG. 9(b) shows the infrared image 2012 acquired bythe second camera 202, and FIG. 9(c) shows an image displayed on thedisplay.

When a specific object 300 exists in an area corresponding to abnormalpixels of the true color image 2011 or in an area corresponding toeffective pixels of the true color image 2011, if it is difficult forthe user to identify the specific object 300, the non-identical stereovision camera may overlap and display a part of the infrared image 212that contains the specific object 300 on the true color image 2011 asshown in FIG. 9(c).

That is, the non-identical stereo vision camera has advantages in thatthe user may easily cope with this situation, that is, the user iscapable of checking the presence of the specific object visually.

The above description is therefore to be construed in all aspects asillustrative and not restrictive. The scope of the present disclosureshould be determined by reasonable interpretation of the appended claimsand all changes coming within the equivalency range of the presentdisclosure are intended to be embraced in the scope of the presentdisclosure.

1-14. (canceled)
 15. An image processing apparatus comprising: camerasconfigured to detect different wavelength bands, wherein: a first cameraamong the cameras is configured to obtain a true color image bycapturing a subject; a second camera among the cameras is configured toobtain an infrared image by capturing the subject, wherein the secondcamera is spaced apart from the first camera; and a controllerconfigured to: match the obtained true color image and the obtainedinfrared image; and obtain three-dimensional information about thecaptured subject in an area corresponding to effective pixels of thematched true color image from the matched infrared image.
 16. The imageprocessing apparatus of claim 15, wherein the effective pixelscorrespond to pixels that absorb light greater than or equal to aminimum amount of light for identifying an object and less than or equalto a saturation amount of light.
 17. The image processing apparatus ofclaim 15, wherein the controller is further configured to coordinate arepresentative pixel of the captured subject among the effective pixelsto the obtained infrared image, wherein the three-dimensionalinformation is obtained based on a stereo vision operation.
 18. Theimage processing apparatus of claim 17, wherein the representative pixelcomprises at least one of an outline pixel of the captured subject or acenter pixel of the captured subject.
 19. The image processing apparatusof claim 15, wherein the controller is further configured to determinewhether a shape of a specific object corresponds to the specific objectbased on information about a related area in the matched infrared imageand based on recognition of the shape of a specific object in the areacorresponding to the effective pixels.
 20. The image processingapparatus of claim 15, wherein the controller is further configured toobtain two-dimensional information about an area corresponding toabnormal pixels of the matched true color image from the matchedinfrared image.
 21. The image processing apparatus of claim 20, whereinthe obtained two-dimensional information comprises information about apresence of a specific object in the area corresponding to the abnormalpixels.
 22. The image processing apparatus of claim 21, wherein thecontroller is further configured to convert the abnormal pixels intoeffective pixels by adjusting at least one of an aperture, an exposuretime, or an ISO of the first camera based on a detection of the presenceof the specific object from the two-dimensional information.
 23. Theimage processing apparatus of claim 22, wherein the controller isfurther configured to obtain three-dimensional information about thespecific object present in the area corresponding to the abnormal pixelsafter adjusting the at least one of the aperture, the exposure time, orthe ISO of the first camera.
 24. The image processing apparatus of claim21, wherein the controller is further configured to: convert theabnormal pixels into the effective pixels based on raw data forobtaining the true color image based on a detection of the presence ofthe specific object from the two-dimensional information; and obtainthree-dimensional information about the specific object present in anarea corresponding to the converted effective pixels.
 25. The imageprocessing apparatus of claim 21, further comprising an outputinterface, wherein the controller is further configured to outputinformation about the specific object to a user through the outputinterface based on a detection of the specific object from thetwo-dimensional information.
 26. The image processing apparatus of claim25, wherein the output interface further comprises a display configuredto display the obtained true color image, and wherein the controller isfurther configured to cause on the display a display of an infraredimage of the specific object in the area corresponding to the abnormalpixels.
 27. The image processing apparatus of claim 15, wherein thefirst camera corresponds to a camera configured to detect visible lightand near-infrared (NIR) light.
 28. The image processing apparatus ofclaim 15, wherein the second camera corresponds to a camera configuredto detect at least one of far-infrared (FIR) light, mid-wave infrared(MWIR) light, or short-wavelength infrared (SWIR) light.